1. Technical Field
The present invention relates to anomaly detection on packet switched communication systems. Particularly, the present invention is related to statistical methods for detecting network traffic anomalies due to network attacks or to communication system failures.
2. Description of the Related Art
Several types of attacks are known, such as: (distributed) denial of service ((D)DoS) attacks, scanning attacks, SPAM or SPIT attacks, and malicious software attacks.
Denial-of-Service (DoS) attacks and, in particular, distributed DoS (DDoS) attacks are commonly regarded as a major threat to the Internet. A DoS attack is an attack on a computer system network that causes a loss of service or network connectivity to legitimate users, that is, unavailability of services. Most common DoS attacks aim at exhausting the computational resources, such as connection bandwidth, memory space, or CPU time, for example, by flooding a target network node by valid or invalid requests and/or messages. They can also cause disruption of network components or disruption of configuration information, such as routing information, or can aim at disabling an application making it unusable. In particular, the network components (e.g., servers, proxies, gateways, routers, switches, hubs, etc.) may be disrupted by malicious software attacks, for example, by exploiting buffer overflows or vulnerabilities of the underlying operating system or firmware.
A DDoS attack is a DoS attack that, instead of using a single computer as a base of attack, uses multiple compromised computers simultaneously, possibly a large or a very large number of them (e.g., millions), thus amplifying the effect. Altogether, they flood the network with an overwhelming number of packets which exhaust the network or application resources. In particular, the packets may be targeting one particular network node causing it to crash, reboot, or exhaust the computational resources. The compromised computers, which are called zombies, are typically infected by malicious software (worm, virus, or Trojan) in a preliminary stage of the attack, which involves scanning a large number of computers searching for those vulnerable. The attack itself is then launched at a later time, either automatically or by a direct action of the attacker.
(D)DoS attacks are especially dangerous for Voice over IP (VoIP) applications, e.g., based on the Session Initiation Protocol (SIP). In particular, the underlying SIP network dealing only with SIP signalling packets is potentially vulnerable to request or message flooding attacks, spoofed SIP messages, malformed SIP messages, and reflection DDoS attacks. Reflection DDoS attacks work by generating fake SIP requests, as an example, with a spoofed (i.e. simulated) source IP address which falsely identify a victim node as the sender, and by sending or multicasting said SIP requests to a large number of SIP network nodes, which all respond to the victim node, and repeatedly so if they do not get a reply, hence achieving an amplification effect.
SPAM attacks consist in sending unsolicited electronic messages (e.g., through E-mail over the Internet), with commercial or other content, to numerous indiscriminate recipients. Analogously, SPIT (SPam over Internet Telephony) attacks consist in sending SPAM voice messages in VOID networks. Malicious software attacks consist in sending malicious software, such as viruses, worms, Trojan, or spyware, to numerous indiscriminate recipients, frequently in a covert manner. Scanning or probing attacks over the Internet consist in sending request messages in large quantities to numerous indiscriminate recipients and to collect the information from the provoked response messages, particularly, in order to detect vulnerabilities to be used in subsequent attacks. For example, in port scanning attacks, the collected information consists of the port numbers used by the recipients.
Attack detection techniques are known which utilize a description (signature) of a particular attack (e.g., a virus, worm, or other malicious software) and decide if the observed traffic data is consistent with this description or not; the attack is declared in the case of detected consistency.
Furthermore, anomaly detection techniques are known which utilize a description (profile) of normal/standard traffic, rather than anomalous attack traffic, and decide if the observed traffic data is consistent with this description or not; an attack or anomalous traffic is declared in the case of detected inconsistency.
Unlike attack detection techniques, anomaly detection techniques do not require prior knowledge of particular attacks and as such are in principle capable of detecting previously unknown attacks. However, they typically have non-zero false-negative rates, in a sense that they can miss to declare an existing attack. They also typically have higher false-positive rates, in a sense that they can declare anomalous traffic in the case of absence of attacks.
Anomaly detection techniques can essentially be classified into two categories: rule-based techniques and statistic-based or statistical techniques. Rule-based techniques describe the normal behavior in terms of certain static rules or certain logic and can essentially be stateless or stateful. In particular, such rules can be derived from protocol specifications.
On the other hand, statistical anomaly detection techniques describe the normal behavior in terms of the probability distributions of certain variables, called statistics, depending on the chosen data features or parameters.
Paper “DDoS detection and wavelets”, L. Li and G. Lee, Telecommunication Systems—Modeling, Analysis, Design and Management, vol. 28, no. 3-4, pp. 435-451, 2005, discloses a method comprising the step of dynamically applying a discrete wavelet transform to overlapping sliding windows of the byte rate curves in time and looking for sudden changes in the logarithms of the associated energy distribution coefficients in order to detect DDoS attacks.
US-A-2004-0220984 describes a method wherein the packet and byte rates are considered as functions of time and, at each time, the mean values and variances of these rates are estimated by using historical data, possibly as Exponentially Weighted Moving Averages (EWMAs), and then a given sample of traffic at a given time is classified by comparing its packet and byte rates with a threshold being proportional to the sum, at the given time, of the historical mean value and the historical standard deviation (i.e., the square root of the variance) multiplied by a positive constant. Anomalous traffic is declared if the threshold is exceeded, i.e., if the observed sample of traffic is classified as an outlier.
U.S. Pat. No. 6,601,014 B1 discloses a method where the mean value and the variance are estimated as the EWMAs, with different, but mutually related associated constants.
Article “EWMA techniques for computer intrusion detection through anomalous changes in event intensity”, N. Ye, C. Borror, and Y. Zhang, Qual. Reliab. Engng. Int., vol. 18, pp. 443-451, 2002, describes a method wherein EWMA techniques are applied for dynamically estimating the mean values and variances of the event intensity process derived from the audit trail data describing the activities on a host machine in a computer network. Anomaly detection is based on the outlier classification principle, where the thresholds are determined under certain probabilistic models for the event intensity process. Alternatively, anomaly detection is based on the estimated variance only, which is compared with a reference value and an alert is then declared if the ratio of the two values is too large or too small.
Paper “Statistical traffic identification method based on flow-level behavior for fair VoIP service”, T. Okabe, T. Kitamura, and T. Shizuno, Proceedings of the 1st IEEE Workshop on VOID Management and Security, Vancouver, Canada, April 2006, pp. 33-38, describes a flow identification method, for VOID media traffic, using the flow statistics such as the minimal and maximal values of the packet inter-arrival time and some characteristics of the packet size distribution comprising the minimal, maximal, average, and median values as well as the total number of different packet sizes occurring in a flow. The statistics are calculated and compared with reference patterns on short time intervals (e.g., 1 second long) and the verification results are averaged over a longer time interval in order to classify a given flow.
Article “Load characterization and anomaly detection for voice over IP traffic”, M. Mandjes, I. Saniee, and A. L. Stolyar, IEEE Transactions on Neural Networks, vol. 16, no. 5, pp. 1019-1026, September 2005, describes a method relating to VOID data traffic that consists in computing the empirical variance estimates of the normalized byte rate on overlapping windows and comparing them with predicted variances that are theoretically obtained under probabilistic models for the number of calls per second. At any time, an anomaly is declared if the ratio of the empirical and theoretical variances is greater than a threshold, which falls in the range between one and two.